Data processing system, data processing method, data processing device, and data processing program

ABSTRACT

An input data is divided into blocks of predetermined time units and then further divided into a plurality of bands. Each band is coded and compressed to prepare a basic sub-block essential for data reproduction and a plurality of extension sub-blocks contributing to improvement of the quality of the reproduced data. A transmission timing during streaming delivery, remaining extension sub-blocks are delivered if the time period is still within a time period for delivering the same data block, but the remaining extension sub-blocks are not delivered and delivery of the basic sub-block of the subsequent data block begins if the delivery timing for the subsequent data block has arrived. In this way, a stable streaming delivery is achieved by a scalable data compression.

TECHNICAL FIELD

The present invention relates to a data processing system fortransmitting and receiving data via a communication line, a dataprocessor and a data processing method for preparing, transmitting, andreceiving data to be distributed, and a data processing program. Morespecifically, the invention relates to a data processing system forstreaming delivery of data via a communication line, a data processorand a data processing method for preparing, transmitting, and receivingdata for stream-delivering, and a data processing program.

Furthermore, the present invention relates to a data processing systemfor streaming delivery of data via a communication line having avariable bandwidth, a data processor and a data processing method forpreparing, transmitting, and receiving data for streaming delivery, anda data processing program. More specifically, the invention relates to adata processing system enabling stable streaming delivery by applying ascalable data compression method, a data processor and a data processingmethod for preparing, transmitting, and receiving data for streamingdelivery by applying a scalable data compression method, and a dataprocessing program.

BACKGROUND ART

Today, data processing and data communication technology are highlydeveloped. As a result, in addition to usual computer data, varioustypes of data such as images (still images and moving images) and audioare processed electronically.

In general, image and audio data contains a large amount of redundantdata and has an extremely large size. Thus, if the image or audio datais stored on a storage device or transmitted through a network in itsoriginal form, the load on the memory capacity and/or the communicationline becomes large. Therefore, when storing or transmitting image oraudio data, the redundancy of the original data is eliminated byencoding the data; in other words, the original data is compressed. Thecompressed image data is decompressed before it is used.

An example of a known high-efficiency coding and compression method foran audio signal is adaptive transform acoustic coding (ATRAC). In ATRAC,the audio signal is divided into four frequency bands by a band-dividingfilter. Then, the divided bands are compressed after the time line datastream is transformed into a frequency axis data stream by modifieddiscrete cosine transform (MDCT). In this case, the absolute thresholdof hearing and the audio masking effect are applied to eliminate audiocomponents that are difficult to hear and the data before and after aloud sound. In this way, the amount of coded data is reduced withoutdegrading the quality of the sound a user hears.

Coding and compression of data, in general, are performed and adjustedso that the data maintains a predetermined data compression rate (bitrate) and a compression-decompression quality. Thus, when data isdelivered via a plurality of lines having different bandwidths or a linehaving a variable transmission quality, the data must be compressed at arate suitable for the bandwidth and/or a plurality of compressed bitstreams having a predetermined quality must be generated.

Recently, a ‘streaming’ technology in which data is reproduced while itis downloaded is widely used on the Internet. However, there is aproblem in that the bandwidth is variable. For example, it is common fora client computer to carry out FIFO buffering to deal with a temporaryreduction in the amount of transferred data or jitter. To respond to alarge change in the amount of transferred data, a plurality of datastreams is prepared by encoding the original data in a plurality of bitrates. Then, the data stream to be delivered is dynamically selected inaccordance with the communication conditions between the server and theclient, and is delivered to the client. An example of the latter case isSure Stream by Real Networks, Inc.

When the above-mentioned methods are used, the first set of data is notreproduced until the FIFO buffer is filled. As a result, there is adelay of several seconds before the streaming begins. When one datastream is selected among a plurality of data streams in accordance withthe communication conditions, the compression process becomes complexbecause a plurality of datasets have to be prepared for one originaldataset. Moreover, managing the system for delivering the data becomescomplicated because extra data storage space and communication lines arerequired to store and transfer the plurality of datasets.

DISCLOSURE OF INVENTION

An object of the present invention is to provide an excellent dataprocessing system for suitably streaming data via a communication linehaving a variable bandwidth, an excellent data processor and a dataprocessing method for preparing, transmitting, and receiving data forstreaming delivery, and a data processing program.

Another object of the present invention is to provide an excellent dataprocessing system for providing a stable streaming delivery using ascalable data compression method, an excellent data processor and a dataprocessing method for preparing, transmitting, and receiving data forstreaming delivery by a scalable data compression method, and anexcellent data processing program.

The present invention has taken into consideration the above-mentionedobjects. A data processing system comprises: a first data processorcomprising transmission data generating means for preparing a basicsub-block essential for reproducing original data and at least oneextension sub-block essential for improving the quality of thereproduced data by dividing the original data into a plurality ofblocks, further dividing each of the plurality of blocks into aplurality of frequency bands, and then compressing the data of theplurality of frequency bands, transmission means for transmitting theprepared basic sub-block and said at least one extension sub-block,timing generating means for generating a timing for transmitting thebasic sub-block and said at least one extension sub-block, andtransmission controlling means for transmitting the basic sub-block of acurrent block based on the timing generated by the timing generatingmeans, for determining, after transmitting the basic sub-block, whethersaid at least one extension sub-block of the current block can betransmitted before a basic sub-block of a subsequent block istransmitted, and for selectively transmitting said at least oneextension sub-block of the current block if it has been determined thatsaid at least one extension sub-block can be transmitted; and a seconddata processor comprising receiving means for receiving a basicsub-block and at least one extension sub-block transmitted from thefirst data processor, reproducing means for reproducing data from thesub-blocks received by the receiving means, and reproduction controllingmeans for reproducing data, if the sub-block received is determined tobe a basic sub-block, by supplying the basic sub-block and said at leastone extension sub-block received by the receiving means before asubsequent basic sub-block is received by the receiving means to thereproducing means.

The term ‘system,’ however, refers to a logical assembly of a pluralityof apparatuses (or functional modules for realizing a predeterminedfunction). Therefore, there are no limitations on whether theapparatuses or functional modules are disposed in a single chassis.

According to the present invention, by applying a scalable datacompression method for dividing the original data into time block units,further dividing these time block units into bands, and then encodingthese bands, datasets having various bit rates can be generated from oneset of data.

Accordingly, the storage capacity for the streaming data required forthe server can be reduced. Since the protocol for controlling the datatransfer rate is simplified, a standard protocol such as a transmissioncontrol protocol (TCP) can be used for stable streaming delivery.Furthermore, since the original data is divided into sub-blocks and theneach sub-block is coded and compressed individually, each compressioncalculation unit can be small (or, in other words, the computationalload of the coding process is small).

For example, when audio data is processed, the original audio data isdivided into an audible frequency band (0 to 22 kHz) and other frequencybands (frequency bands exceeding 22 kHz may be divided into a pluralityof frequency bands) for each time block. Then, each frequency band iscoded and compressed. Coding and compression are carried out by modifieddiscrete cosine transform (MDCT) and quantization. In the process ofcoding and compression, compression components and quantization errorcomponents are output. The compression component for audio frequency isessential for data reproduction and is assigned as a basic sub-block.The quantization error component for audio frequency, and thecompression components and the quantization components of otherbandwidths contribute to an improvement in the quality of the datareproduced and are assigned as extension sub-blocks.

Depending on the characteristics of the data, the data of a quantizationerror component may be large. In such a case, the coding and compressionprocess may be applied recursively to the quantization error component.In this way, compression components and quantization error componentsare further generated from the quantization error component. Thesecompression and quantization error components may be included in thestream data as extension sub-blocks.

In this way, data is processed by dividing the data into a plurality ofsub-blocks having a hierarchical structure in the predetermined timeunits. Resource amount calculation information may be added to eachsub-block for determining the amount of resource required fortransferring the sub-block. The resource amount calculation informationis used for calculating the incremental block transfer rate due to thetransfer of sub-blocks and, for example, is the data length indicatingthe size of the sub-block. Instead of data length, difference in theblock transfer rate for when the sub-block is and is not transferred maybe used. For example, a hierarchy ID for identifying the time block andband the sub-block belongs to may be added in addition to the datalength of the sub-block to each sub-block.

In streaming delivery, the first data processor that delivers the databy functioning as a stream server transmits the basic sub-block of eachpredetermined time unit without fail whereas the number of extensionsub-blocks to be delivered is dynamically controlled in accordance withthe bandwidth. In other words, when a transmission timing is detected,the data block to be transmitted is determined. For example, if thedetected transmission timing arrives within a time period fortransmitting the same data block, the remaining extension sub-blocks maybe transmitted. If, however, the detected transmission timing arriveswithin a time period for transmitting the subsequent data block, theremaining sub-blocks of the previous data blocks are not transmittedand, instead, the basic sub-block for the subsequent data block istransmitted. Here, the term ‘timing for transmission’ refers to a timewhen a subsequent dataset can be delivered through a transmission lineor a time equivalent. For example, the time when a free space isgenerated in a protocol buffer used for absorbing the delivery jittermay be used as the timing for transmission.

The second data processor for receiving a data stream that is the clientcarries out streaming reproduction by decoding and decompressing thebasic sub-block for each time block by inverse-quantization andinverse-MDCT and uses the extension sub-blocks to improve the quality ofthe reproduced data. The receiver is not aware of how many extensionsub-blocks per time block are delivered, but, by identifying thesub-blocks according to their hierarchy IDs, the data can be reproduced.Moreover, the receiver can refer to the hierarchy IDs, as required, toselect or refuse sub-blocks in predetermined hierarchal levels. Forexample, an amount of data indicated by a data length can be skipped inthe data stream to eliminate unwanted extension sub-blocks. In otherwords, the receiver can select the quality of decompression for decodingand decompressing the received data. Since each sub-block is decoded anddecompressed individually, each compression calculation unit can besmall (or, in other words, the computational load of the coding processis small).

A data processing method according to a second aspect of the presentinvention comprises the steps of generating a basic sub-block essentialfor reproducing original data and extension sub-blocks essential forimproving the quality of the reproduced data by dividing the originaldata into a plurality of blocks, further dividing each of the pluralityof blocks into a plurality of frequency bands, and then compressing oneof the plurality of frequency bands, and transmitting the basicsub-block of a current block based on the timing generated by a timinggenerating means, determining, after transmitting the basic sub-block,whether the extension sub-blocks of the current block can be transmittedbefore a basic sub-block of a subsequent block is transmitted, andselectively transmitting the extension sub-blocks of the current blockif it has been determined that the extension sub-blocks can betransmitted.

According to the data processing method according to the second aspectof the present invention, scalable hierarchically-compressed data isobtained. From this data, blocks of data having various bit rates areproduced. Streaming delivery of such hierarchically-compressed datareduces the storage capacity on the server used for the streaming data.

Since the protocol for controlling the data transfer rate is simplified,a standard protocol such as a transmission control protocol (TCP) can beused to provide a stable streaming delivery through the Internet, whichhas a variable bandwidth.

Furthermore, since each sub-block is decoded and decompressedindividually, each compression calculation unit can be small (or, inother words, the computational load of the coding process is small).

According to the transmission data generating means or the step forgenerating transmission data, data of a time block unit is divided intoa band essential for data reproduction and at least one other band, acalculation for coding and compression is applied to each band, acompression component of the band essential for data reproduction isassigned as a basic sub-block, and compression components of the bandsexcepting the essential band and compression error components of all thebands are assigned as extension sub-blocks.

According to the transmission data generating means or the step forgenerating transmission data, the compression error components for allbands may be recursively coded and compressed and the recursivecompression components and the recursive compression error componentsmay be assigned as extension sub-blocks.

In this way, input data is divided into blocks of predetermined timeunits and then further divided into a plurality of bands. Each band iscoded and compressed to prepare a hierarchical structure of a basicsub-block essential for data reproduction and a plurality of extensionsub-blocks not essential for data reproduction but that contribute toimprovement of the quality of the reproduced data.

The hierarchically-compressed data is scalable. Therefore, when thecompressed data is decompressed, the data stream can be selectivelydecompressed entirely or partially to select and adjust the quality ofthe decompressed data.

According to the transmission data determining means or the step fordetermining the transmission data, hierarchy IDs and data lengths may beadded to the sub-blocks as resource amount calculation information fordetermining the amount of resource required for transferring thesub-blocks. Instead of data length, difference in the data lengths maybe added. Based on the resource amount calculation information, theincremental block transfer rate due to the transfer of sub-blocks can becalculated. Consequently, when scalable compressed data is recorded,read out, or transmitted, the subsequent sub-block can be read outquickly by identifying the sub-block based on the hierarchy ID and byskipping an amount of data indicated by the data length and forwardingto a new readout position.

According to the transmission data determining means or the step ofdetermining the transmission data, the transmission of a basic sub-blockfor a time block is determined and an extension sub-block to betransmitted is selected in accordance with the bandwidth during datastreaming delivery.

For example, if the detected transmission timing is in the same timeblock as the last sub-block delivered or, in other words, if there isstill some remaining time for transmitting data in the time block inwhich the last data transmission was performed, whether the remainingextension sub-blocks in the same time block are transmitted isdetermined. As a result, in addition to the basic sub-block essentialfor data reproduction, a number of extension sub-blocks in accordancewith the bandwidth may be transmitted. Thus, the client receiving thedata can perform high-quality data reproduction.

If, at the transmission timing detected, the transmission period in thesame time block as that in which the last sub-block was delivered hasalready elapsed, it is determined to transmit the basic sub-block of thetime block corresponding to the detected transmission timing. As aresult, if there is no leeway in the bandwidth, at least the basicsub-block is transmitted to avoid interruption of the reproduced data atthe client that receives the data.

A data processing method according to a third aspect of the presentinvention comprises the steps of receiving a basic sub-block andextension sub-blocks, and reproducing data, if the sub-block received isdetermined to be a basic sub-block, from the basic sub-block and theextension sub-blocks received before a subsequent basic sub-block isreceived.

According to the data processing method according to the third aspect ofthe present invention, interruption of the reproduced data is avoided byreproducing the data using at least a basic sub-block if there is noleeway in the bandwidth. If there is leeway in the bandwidth, anextension sub-block is selected from the plurality of extensionsub-blocks received before a subsequent basic sub-block is received andthen data is reproduced from the selected extension sub-blocks and thebasic sub-block. In this way, high-quality data reproduction ispossible.

A data processing program according to a fourth aspect of the presentinvention written in a language readable by a computer so that data canbe processed on a computer system, comprises the steps of dividing thedata into a plurality of blocks of a predetermined time unit, generatinga basic sub-block essential for reproducing original data and at leastone extension sub-block essential for improving the quality of thereproduced data by dividing the original data into a plurality ofblocks, further dividing each of the plurality of blocks into aplurality of frequency bands, and then compressing one of the pluralityof frequency bands, and transmitting a basic sub-block of apredetermined time unit, determining, after transmitting the basicsub-block, whether said at least one extension sub-block of the currentblock can be transmitted before a basic sub-block of a subsequent blockis transmitted, and transmitting said at least one extension sub-blockif it has been determined that said at least one extension sub-block canbe transmitted.

A data processing program according to a fifth aspect of the presentinvention written in a language readable by a computer so that data canbe processed on a computer system, comprises the steps of determiningwhether the received sub-block is a basic sub-block and reproducingdata, if the received sub-block is a basic sub-block, from the basicsub-block and at least one extension sub-block received before asubsequent basic sub-block is received.

The computer program according to the fourth and fifth aspect of thepresent invention defines a computer program written in a languagereadable by a computer. In other words, by installing the computerprogram according to the fourth and fifth aspect of the presentinvention, a cooperative operation is carried out on the computer systemand results similar to the data processing method according to thesecond and third aspects are achieved.

Other objects, characteristics, and advantages of the present inventionwill be apparent through the detailed descriptions based on theembodiments and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of the data compression mechanism used forstreaming delivery according to an embodiment of the present invention.

FIG. 2 is a schematic view of the data structure of the original dataread out from a recording apparatus.

FIG. 3 is a schematic view of the hierarchical sub-block structureobtained by coding and compression by the data compression mechanismillustrated in FIG. 1.

FIG. 4 illustrates the coding and compression mechanism for recursivelycompressing the quantization error components.

FIG. 5 illustrates the structure of the hierarchically-compressed dataincluding recompression components and requantization error componentsof an extension sub-block.

FIG. 6 illustrates the structure of the hierarchically-compressed dataincluding compressed-recompression components andquantized-requantization error components of an extension sub-block.

FIG. 7 is a schematic view of the data structure of the basic andextension sub-block.

FIG. 8 is a schematic view illustrating the scalablyhierarchically-compressed data stored in a storage apparatus.

FIG. 9 is a schematic view illustrating a streaming delivery system 50according to an embodiment of the present invention.

FIG. 10 is a schematic view illustrating the functional structure of thestreaming delivery process carried out by a streaming delivery serverapplication 62.

FIG. 11 is a flow chart indicating the steps of a streaming deliveryprocess carried out by the streaming delivery server application 62.

FIG. 12 is a flow chart indicating the steps of receiving streamingdelivery data by a client application 7 for reproducing streamingdelivery data.

FIG. 13 is a schematic view illustrating the structure of a datadecoding and decompression system 20 capable of performing a scalabledata coding and compression.

FIG. 14 illustrates the results of the decoding and decompressionprocess performed on the scalably compressed data by the data decodingand decompression system 20.

FIG. 15 illustrates the results of the decoding and decompressionprocess performed on the scalably compressed data by the data decodingand decompression system 20.

FIG. 16 illustrates the results of the decoding and decompressionprocess performed on the scalably compressed data by the data decodingand decompression system 20.

FIG. 17 illustrates the results of the decoding and decompressionprocess performed on the scalably compressed data by the data decodingand decompression system 20.

FIG. 18 illustrates the results of the decoding and decompressionprocess performed on the scalably compressed data by the data decodingand decompression system 20.

FIG. 19 illustrates the results of the decoding and decompressionprocess performed on the scalably compressed data by the data decodingand decompression system 20.

FIG. 20 illustrates the results of the decoding and decompressionprocess performed on the scalably compressed data by the data decodingand decompression system 20.

FIG. 21 illustrates the results of the decoding and decompressionprocess performed on the scalably compressed data by the data decodingand decompression system 20.

FIG. 22 illustrates the results of the decoding and decompressionprocess performed on the scalably compressed data by the data decodingand decompression system 20.

FIG. 23 illustrates the results of the decoding and decompressionprocess performed on the scalably compressed data by the data decodingand decompression system 20.

FIG. 24 is a schematic view of the structure of the data streamdelivered from a server 60.

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiments of the present invention will now be described in detail byreferring to the drawings.

A. Overview

When data streaming is performed by using a communication line having avariable bandwidth, such as the Internet, a scalable data compressionmethod that can be adapted to the various compression rates inaccordance with the bandwidth is required.

Accordingly, in the present invention, data is divided into blocks ofpredetermined time units that are further divided into a plurality ofbands. Then, each band is coded and compressed to prepare a hierarchicalstructure including a basic sub-block that is essential for thereproduction of the data and a plurality of extension sub-blocks thatare not essential for data reproduction but contribute to theimprovement of the quality of the reproduced data.

In streaming delivery, the basic sub-block of each predetermined timeunit is transmitted without fail whereas the number of extensionsub-blocks to be delivered is dynamically controlled in accordance withthe bandwidth. In other words, when a transmission timing is detected,the data block to be transmitted is determined. For example, if thedetected transmission timing arrives within a time period fortransmitting the same data block, the remaining extension sub-blocks maybe transmitted. If, however, the detected transmission timing arriveswithin a time period for transmitting the subsequent data block, theremaining sub-blocks of the previous data blocks are not transmittedand, instead, the basic sub-block for the subsequent data block istransmitted.

By applying a scalable data compression method for dividing the originaldata into time block units, further dividing these time block units intobands, and then encoding these bands, datasets having various bit ratescan be generated from one set of data. Accordingly, the storage capacityfor the streaming data required for the server can be reduced. Since theprotocol for controlling the data transfer rate is simplified, astandard protocol such as a transmission control protocol (TCP) can beused for stable streaming delivery. Furthermore, since the original datais divided into sub-blocks and then each sub-block is coded andcompressed individually, each compression calculation unit can be small(or, in other words, the computational load of the coding process issmall).

For example, when audio data is delivered by streaming, the originalaudio data is divided into an audible frequency band (0 to 22 kHz) andother frequency bands (frequency bands exceeding 22 kHz may be dividedinto a plurality of frequency bands) for each time block. Then, eachfrequency band is coded and compressed. Coding and compression arecarried out by modified discrete cosine transform (MDCT) andquantization. In the process of coding and compression, compressioncomponents and quantization error components are output. The compressioncomponent for audio frequency is essential for data reproduction and isassigned as a basic sub-block. The quantization error component foraudio frequency, and the compression components and the quantizationcomponents of other bandwidths contribute to an improvement in thequality of the data reproduced and are assigned as extension sub-blocks.

Depending on the characteristics of the data, the data of a quantizationerror component may be large. In such a case, the coding and compressionprocess may be applied recursively to the quantization error component.In this way, compression components and quantization error componentsare further generated from the quantization error component. Thesecompression and quantization error components may be included in thestream data as extension sub-blocks.

In this way, data is divided into a hierarchical structure of aplurality of sub-blocks in predetermined time units. Each sub-blockincludes a hierarchy ID for indicating which time block and band thesub-block belongs to and a data length for indicating the size of thesub-block.

A client that is the receiver of a data stream carries out streamingreproduction by decoding and decompressing the basic sub-block for eachtime block by inverse-quantization and inverse-MDCT and uses theextension sub-blocks to improve the quality of the reproduced data. Thereceiver is not aware of how many extension sub-blocks per time blockare delivered, but, by identifying the sub-blocks according to theirhierarchy IDs, the data can be reproduced. Moreover, the receiver canrefer to the hierarchy IDs, as required, to select or refuse sub-blocksin predetermined hierarchal levels. For example, an amount of dataindicated by a data length can be skipped in the data stream toeliminate unwanted extension sub-blocks. In other words, the receivercan select the quality of decompression for decoding and decompressingthe received data. Since each sub-block is decoded and decompressedindividually, each compression calculation unit can be small (or, inother words, the computational load of the coding process is small).

B. Scalable Data Compression

A scalable data coding and compression system 10 according to thepresent invention will now be described by referring to FIG. 1. Forconvenience, the data processed in the description below is audio data.Furthermore, the audio data will have a frequency band of no more than88.2 kHz. In other words, the audio data has a sampling rate of 176.4kHz.

The original data is stored on a recording apparatus that can berandomly accessed, such as a hard disk or a magnetic-optical disk. Theaudio data is read out from the recording apparatus in predeterminedtime block T_(B) units (refer to FIG. 2) and then divided into aplurality of sub-blocks in the frequency direction by a multiplex ofquadrature mirror filters (QMFs).

The embodiment illustrated in FIG. 1 divides the audio data intosub-blocks in an audible frequency band of 0 to 22 kHz and frequencybands over 22 kHz with increments of 22 kHz by a two-stage QMF 11.Hereinafter, the audible frequency band is referred to as a ‘basicband,’ each band in 22-kHz increments exceeding 22 kHz will be referredto as an ‘x2 band,’ an ‘x3 band,’ and an ‘x4 band,’ in order. Obviously,the frequency band exceeding 22 kHz may be divided into differentincrements or may be treated as one sub-block in a single band withoutbeing divided.

To code and compress the divided bands, the time line data streams aretransferred into frequency line data streams by a modified discretecosine transform (MDCT) calculating unit 12 and then are quantized by aquantization calculating unit 13.

In this way, the output of the quantization calculating unit 13 for eachband is composed of a compression component and a quantization errorcomponent.

As a result, as illustrated in FIG. 3, the original data sectioned intopredetermined time blocks are further divided into a plurality ofhierarchized sub-blocks. More specifically, the basic band has ahierarchical structure including a basic sub-block #1 composed of acompression component of the basic band and an extension sub-block #2composed of a quantization error component of the basic band. The x2band has a hierarchical structure including a basic sub-block #3composed of a compression component of the x2 band and an extensionsub-block #4 composed of a quantization error component of the x2 band.The x3 band has a hierarchical structure including a basic sub-block #5composed of a compression component of the x3 band and an extensionsub-block #6 composed of a quantization error component of the x3 band.The x3 band has a hierarchical structure including a basic sub-block #5composed of a compression component of the x3 band and an extensionsub-block #6 composed of a quantization error component of the x3 band.In this embodiment, the sub-blocks have variable lengths. Thesub-blocks, however, may also be coded and compressed so that they havea fixed length.

Depending on the characteristics of the data, the data of a quantizationerror component may be large. In such a case, as illustrated in FIG. 4,the coding and compression process may be applied recursively to thequantization error component by a recompression unit 14. In this way,the recompression unit 14 further generates a recompression componentand a requantization error component from the quantization errorcomponent. As illustrated in FIG. 5, these recompression andrequantization error components may be included in a stream data asextension sub-blocks. Accordingly, as illustrated in FIG. 6, theextension sub-blocks can be further hierarchized by obtaining acompressed-recompression component and a quantized-requantization errorcomponent by further compressing the requantization error component.

The basic sub-block #1 is a component essential for reproducing thedata. On the other hand, extension sub-blocks #2 to #8 are not essentialfor reproducing the data but contribute to an improvement in the qualityof the data reproduction. There are two types of extension sub-blocks: acompression component of the original data and a quantization errorcomponent of the original data. However, without thecorresponding-compression component, the quantization error componentcan not be used for data reproduction. (For example, to use theextension sub-block #4, the extension sub-block #3 must be decoded). Asa band component is closer to the basic sub-block #1, the moresignificant is its contribution to the data reproduction.

At a post-processor 15, a hierarchy ID is added to the hierarchicallycoded and compressed sub-blocks to indicate which time block and bandeach of the sub-blocks belongs to. When a sub-block has a variablelength, a data length will be added to the sub-block. FIG. 7 illustratesthe data structure of a basic sub-block and an extension sub-block.

The basic sub-blocks and the extension sub-blocks generated in such amanner are stored on a randomly accessible recording apparatus such as ahard disk device as time-series data (refer to FIG. 8). Which time blockand band a sub-block belongs to can be identified for each sub-blockstored on the recording apparatus by referring to the hierarchy ID addedto the head of the sub-block. To proceed to a subsequent sub-block, thereadout position is forwarded by skipping an amount of data indicated bythe data length.

By dividing the compressed data and hierarchically recording thecompressed data, the quality of the decompressed data is dynamicallyselected and controlled, realizing scalability of the compressed data.The quality of the compressed data is improved by detecting errorsgenerated in the compression process and adding this information to thecompressed data.

FIG. 13 illustrates a data decoding and decompression system 20corresponding to the above-described scalable data coding andcompression. As illustrated in FIG. 13, the decompression system 20includes a decoding and decompression unit for each of the audible, x2,x3, and x4 bands. The decompression system 20 multiplexes the decodingdata for each band by inverse QMF to reproduce the original data.

A post-processor 21 reads the hierarchy ID of the received sub-block andsends the sub-block to a decoding and decompression unit of thecorresponding band.

The decoding and decompression unit includes an inverse-quantizationunit 22 and an inverse-MDCT calculating unit 23. Theinverse-quantization unit 22 performs inverse-quantization on thecompression component sub-block and the quantization error componentsub-block of the corresponding band. If the quantization error sub-blockhas been recursively compressed, the sub-block is first decoded anddecompressed and then sent to the inverse-quantization unit 22. Theinverse-MDCT calculating unit 23 transfers a frequency line data streaminto a time line data stream.

The decoded data for each band is multiplexed by an inverse-QMF 24 toreproduce the original data.

The basic sub-block #1 is input into the data decoding and decompressionsystem 20, and, then, inverse-quantization and inverse-MDCT are carriedout to reproduce the audible frequency band of the original data (referto FIG. 14). Subsequently, the extension sub-block #2 is input, and,then, the quantization error component generated when the audiblefrequency band was coded and compressed is reproduced (refer to FIG.15). In this way, the audible frequency band is reproduced inhigh-quality.

The extension sub-block #3 is input and the x2 band is reproduced (referto FIG. 16). Subsequently, the extension sub-block #4 is input, and,then, the quantization error component generated when the x2 band wascoded and compressed is reproduced (refer to FIG. 17). In this way, thebands up to the x2 band are reproduced in high-quality.

The data decoding and decompression system 20 obtains the extensionsub-blocks #5, #6, . . . in sequence to reproduce the higher bandcomponents of the original data in high quality (refer to FIGS. 18 to21).

As a variation of a data decoding and decompression process forgenerating scalable compressed data, as illustrated in FIG. 21, thecompression and coding components for all bands (audible band and x2,x3, and x4 bands) may be reproduced while the quantization errorcomponent for only the audible band is reproduced. As another variation,as illustrated in FIG. 22, the compression and coding components for allbands (audible band and x2, x3, and x4 bands) may be reproduced whilethe quantization error components for only the audible band and the x2band are reproduced. As another variation, as illustrated in FIG. 23,the compression coding components for the audible band and the x2 and x3bands may be reproduced while the quantization error component for onlythe audible band is reproduced. The post-processor 21 refers to thehierarchy ID and the data length added to each sub-block supplied to thedata decoding and decompression system 20 to select sub-blocks so thatthe decoding and decompression results illustrated in FIGS. 22 and 23are obtained.

As described above, the compressed data is scalable. Therefore, whendecompressing the compressed data, the entire data stream and aselective part of the data stream can be decompressed to select andadjust the quality of the decompressed data.

C. Streaming Delivery

According to the data coding and compression system 10 illustrated inFIG. 1, scalable hierarchically-compressed data is obtained. From thisdata, blocks of data having various bit rates are produced. Streamingdelivery of such hierarchically-compressed data reduces the storagecapacity on the server used for the streaming data. Since the protocolfor controlling the data transfer rate is simplified, a standardprotocol such as a transmission control protocol (TCP) can be used toprovide a stable streaming delivery through the Internet, which has avariable bandwidth.

FIG. 9 is a schematic view illustrating the structure of the streamingdelivery system 50 according to this embodiment.

As illustrated in FIG. 9, the streaming delivery system 50 includes aserver 60 for delivering the data streams and a client 70 for receivingthe data streams as the data is reproduced simultaneously.

The server 60 and the client 70 are interconnected via the Internet orother networks such as a TCP network 80. In this embodiment, the TCPnetwork 80 does not necessarily have to have a constant bandwidth inorder to carry out a stable streaming delivery.

Although not illustrated in the drawing, a plurality of servers andclients may be connected to the TCP network 80. In addition to thestreaming delivery service, the server may provide other variouscommunications services.

On the server 60 providing streaming delivery, a streaming deliveryserver application 62 operates on a TCP protocol stack 61.

The server application 62 reads out the streaming delivery data from anannexed recording apparatus 63 and performs streaming delivery via theTCP protocol stack 61. For details on a TCP protocol, refer to theRequest For Comments (RFC).

Streaming delivery data that has been scalably compressed is stored onthe recording apparatus 63. More specifically, the original data isdivided into a plurality of bands for each predetermined time block.Then, the data is hierarchically coded and compressed into compressioncomponent sub-blocks and quantization component sub-blocks. Thesub-blocks are stored in a time series (refer to FIG. 8). Each sub-blockincludes a hierarchy ID for indicating which time block and band thesub-block belongs to and a data length for indicating the size of thesub-block (refer to FIG. 7). The recording apparatus 63 includes arandomly accessible apparatus such as a hard disk that allows freeaccess to any sub-block by referring to the hierarchy ID and the datalength.

The streaming delivery server application 62 reads out a sub-block to bedelivered next from the recording apparatus 63 and then writes thissub-block onto a transmission buffer 61A included in the TCP protocolstack 61.

The transmission buffer 61A applies a First In First Out (FIFO) methodto deliver the data in chronological order to the TCP network 80. Thetransmission buffer 61A is installed to compensate for the difference inthe timing for reading out the transmission data from the recordingapparatus 63 (or the timing for generating the transmission data) andthe timing for transmitting the data. The data transmission timing forthe TCP protocol stack 61 arrives when the transmission data is writtenin the transmission buffer 61A (or when the transmission buffer 61Abecomes full) by a writing operation of the application 62. When thetransmission buffer 61A becomes empty (or when free space is generated)as data is transmitted to the TCP network 80, the subsequent timing forthe writing operation of the application 62 arrives.

On the client 70 receiving the streaming data, a streaming datareproduction client application 72 operates on a TCP.

When the TCP protocol stack 71 receives the data delivered via the TCPnetwork 80, the TCP protocol stack 61 writes this data into a receivebuffer 71A. The streaming data reproduction client application 72reproduces the original data by performing inverse-quantization andinverse-MDCT to the data read out from the receive buffer 71A and thenoutputs the reproduced data from a reproducer 73. This decoding processis carried out by using the compression component sub-blocks for eachband and the quantization error component sub-blocks.

As described above, the streaming delivery data that has been scalablycompressed is transmitted from the server 60. More specifically, thetransmission data is prepared by dividing the original data into aplurality of bands for each predetermined time block and then coded andcompressed. Subsequently, the compression component sub-blocks and thequantization component sub-blocks are hierarchically arranged. Theclient 70 is not aware of how many extension sub-blocks of each of thetime blocks have been delivered, but, by identifying the sub-blocksaccording to their hierarchy IDs, the data can be reproduced.

The receive buffer 71A applies a FIFO method to deliver the data inchronological order to the application 72. The receive buffer 71A isinstalled to compensate for the difference in the timing for receivingthe reception data via the TCP network 80 (or the timing for reproducingthe reception data) and the timing for reading out the reception datafrom the application 72. When the reception data is stored in thereceive buffer 71A (or when the receive buffer 71A becomes full) theapplication 72 reads out the reception data. When the receive buffer 71Abecomes empty (or when free space is generated) by the readout operationof the application 72, a timing for the TCP protocol stack 71 to receivesubsequent data arrives.

In this embodiment, the original data is provided in a hierarchicalstructure including a basic sub-block that is essential for datareproduction and a plurality of extension sub-blocks that is notessential for data reproduction but contribute to an improvement in thequality of the reproduced data. The streaming delivery serverapplication 62 transmits the basic sub-blocks without fail anddynamically controls the number of extension sub-blocks to be deliveredin accordance with the bandwidth. More specifically, if a timing fortransmission arrives during a time period for delivering sub-blocksbelonging to the same data block as the data block thepreviously-delivered sub-blocks belonged to, the remaining extensionsub-blocks of the data block are delivered, but if a timing fortransmission arrives during a time period for delivering a subsequentdata block, the remaining extension sub-blocks are not delivered and thebasic sub-block of the subsequent data block is delivered.

In this way, various bit rates can be generated from one set of data byapplying a scalable data compression method for further dividing andcoding the original data into bands according to the time block units.Since a plurality of coded streams having different bit rates do nothave to be prepared from one set of original data to adjust the bitrate, the storage capacity of the recording apparatus 63 can be reduced.

FIG. 10 is a schematic view illustrating the functional structure of thestreaming delivery process carried out by the streaming delivery serverapplication 62.

As illustrated in the drawing, the streaming delivery server application62 includes a timer 65, a delivery block determining unit 66, and adelivered block storage unit 67.

To carry out streaming delivery in which downloading and reproducing ofdata is performed simultaneously, the bit rate must be dynamicallyadjusted by the server 60 to enable uninterrupted data reproduction atthe client without being affected by a change in the bandwidth.

Data to be transmitted is divided into a plurality of bands for eachpredetermined time block and is further divided into hierarchicallyarranged delivery blocks of a basic sub-block that is essential to datareproduction and a plurality of extension sub-blocks that contribute tothe improvement of the quality of the reproduced data. Thus, to avoidinterruption in the data, the basic sub-blocks for each predeterminedtime block must be delivered. Moreover, to improve the quality of thereproduced data, the extension sub-blocks are delivered depending on theleeway in the bandwidth.

The timer 65 measures the actual time by a system timer (not depicted inthe drawing).

The delivery block determining unit 66 refers to the timing value of thetimer 65 and calculates the time block for when the transmitting timingwas detected. If a timing for transmission arrives during a time periodfor delivering sub-blocks belonging to the same data block as the datablock the previously-delivered sub-blocks belonged to, the remainingextension sub-blocks of the data block are delivered, but if a timingfor transmission arrives during a time period for delivering asubsequent data block, the remaining extension sub-blocks are notdelivered and the basic sub-block of the subsequent data block isdelivered. To determine which block to deliver, the delivery blockdetermining unit 66 uses the delivered block storage unit 67 to maintainthe hierarchy ID of the block B_(last) and the sub-block SB_(last) thatwere delivered last.

The delivery block determining unit 66 determines the block and thesub-block to be delivered by the above-mentioned mechanism. Then, thedelivery block determining unit 66 reads out the applicable data fromthe recording apparatus 63 and writes this data into the transmissionbuffer 61A of the TCP protocol stack 61.

FIG. 11 is a flow chart describing the streaming delivery process by thestreaming delivery server application 62.

The server application 62 checks the free space in the transmissionbuffer 61A of the TCP protocol stack 61 and determines whether theoutput timing has arrived (Step S1).

If the output timing has arrived, the server application 62 refers tothe timing value of the timer 65 and calculates the current time blockB_(current). Also, the server application 62 refers to the deliveredblock storage unit 67 to detect the block B_(last) that was deliveredlast. The server application 62 determines whether the block B_(last)that was delivered last matches the current time block B_(current) (StepS2).

If the block B_(last) that was delivered last matches the current timeblock B_(current), the sub-block SB_(last) that was delivered laststored in the delivered block storage unit 67 is read out. Then, theserver application 62 checks whether a subsequent sub-block SB_(last)+1is present in the same time block B_(last) (Step S3).

If a subsequent sub-block SB_(last)+1 is not present in the same timeblock B_(last), the process returns to Step S1 since all the sub-blocksin the time block B_(last) have already been transmitted. Then, theserver application 62 waits for the data transmission timing for thenext time block.

If a subsequent sub-block SB_(last)+1 is present in the same time blockB_(last), the subsequent sub-block SB_(last)+1 is selected in the sametime block B_(last) as output data (Step S4). The selected sub-block isread out from the recording apparatus 63 and is written in thetransmission buffer 61A of the TCP protocol stack 61 (Step S6). Then,the delivered block storage unit 67 updates the content of the datastored in the delivered block storage unit 67.

If the block B_(last) that was delivered last does not match the currenttime block B_(current) in Step S2, the basic sub-block of the currenttime block B_(current) is selected as output data since the time blockfor transmitting the block B_(last) that was delivered last has alreadyelapsed (Step S5). The selected sub-block is read out from the recordingapparatus 63 and is written in the transmission buffer 61A of the TCPprotocol stack 61 (Step S6). At the same time, the content of the storeddata is updated.

Subsequently, the process returns to Step S1 and the streaming deliveryserver application 62 waits for a subsequent data transmission timing.

FIG. 24 is a schematic view of the structure of the data streamsdelivered from the server 60. In this embodiment, the streaming deliverydata is processed per each predetermined time block T_(B). The number ofsub-blocks to be delivered is determined according to the bandwidth forthe time block. As a result, the bit rate of the data stream isdynamically adjusted and controlled. According to the embodimentillustrated in FIG. 24, in the time block for t=0, the basic sub-blockof hierarchy #1 and extension sub-blocks of hierarchy #2 and #3 aredelivered. In the time block for t=T_(B), the basic sub-block ofhierarchy #1 and only the extension sub-block of hierarchy #2 aredelivered.

As described above, the streaming delivery system 50 according to thisembodiment can easily prepare datasets having various bit rates inaccordance with the bandwidth in each time block T_(B) from one set ofdata such as the dataset illustrated in FIG. 8. Accordingly, theprotocol for controlling the bit rate is simplified, so that, even whenthe streaming delivery system 50 is installed on a standard TCP,sufficiently stable streaming delivery is possible. Moreover, thestorage capacity of the recording apparatus 63 of the server 60 can bereduced.

The streaming data reproduction client application 72 reproduces theoriginal data by carrying out inverse-quantization and inverse-MDCT onthe data read out from the receive buffer 71A and then outputs the datafrom the reproducer 72. FIG. 12 is a flow chart indicating the receptionprocess of the streaming data by the streaming data reproduction clientapplication 72.

The client application 72 reads the hierarchy ID in the head of thesub-block read out from the receive buffer 71A (Step S11).

Then, based on the hierarchy ID, the client application 72 determineswhether the read out sub-block is the basic sub-block for the applicabletime block (Step S12).

If the read out sub-block is a basic sub-block, the sub-blocks in thebuffers of the decoding and decompressing units of each hierarchy makeup all the data available for a data block. Therefore, the data in eachbuffer is decoded. Then, after the decoded data is sent out from thereproducer 73 (Step S13), the receive buffer 71A is cleared (Step S14).Subsequently, the data of the basic sub-block is read in the same manneras sub-blocks of other hierarchies (Step S15).

If the read sub-block is not a basic sub-block, the sub-blocks of theapplicable hierarchy are read into the input buffers of the decoding anddecompression unit of each hierarchy in sequence (Step S15).

Addendum

In the above, the present invention has been described in detail byreferring to a particular embodiment. It is, however, apparent thatanyone skilled in the art may apply modifications and substitutions tothe particular embodiment without going beyond the scope of the presentinvention. In other words, the present invention is not limited to thedisclosed embodiments included in the specification. To understand thecontent of the present invention, the claims stated earlier must bereferred to.

INDUSTRIAL APPLICABILITY

According to the present invention, an excellent data processing systemfor streaming delivery of data via a communication line having avariable bandwidth, an excellent data processor and a data processingmethod for preparing, transmitting, and receiving data for streamingdelivery, and a computer program are provided.

According to the present invention, an excellent data processing systemfor a stable streaming delivery using a scalable data compressionmethod, an excellent data processor and a data processing method forpreparing, transmitting, and receiving data for streaming delivery by ascalable data compression method, and a computer program are provided.

According to the present invention, a single apparatus or an informationcompression method can be efficiently used at a wide-range ofcompression ratios. In particular, when the bandwidth of a transmissionline is not stable, data having a compression ratio corresponding to thebandwidth can be automatically prepared.

According to the present invention, a compressed bit stream cancorrespond to a plurality of bit rates. Thus, the same compressed bitstream can be used for transferring information through transmissionlines having various transfer rates and recording information onrecording media having different recording capacities (densities).Therefore, a plurality of bit streams having different compressionratios that correspond to the transfer lines and recording media doesnot have to be provided. Consequently, a standard for a system can beeasily established and the size of the apparatus can be reduced.

1. A data processing system comprising: a first data processorcomprising: transmission data generating means for preparing a basicsub-block essential for reproducing original data and at least oneextension sub-block essential for improving the quality of thereproduced data by dividing the original data into a plurality ofblocks, further dividing each of the plurality of blocks into aplurality of frequency bands, and then compressing the data of theplurality of frequency bands; transmission means for transmitting theprepared basic sub-block and said at least one extension sub-block;timing generating means for generating a timing for transmitting thebasic sub-block and said at least one extension sub-block; andtransmission controlling means for transmitting the basic sub-block of acurrent block based on the timing generated by the timing generatingmeans, for determining, after transmitting the basic sub-block, whethersaid at least one extension sub-block of the current block can betransmitted before a basic sub-block of a subsequent block istransmitted, and for selectively transmitting said at least oneextension sub-block of the current block if it has been determined thatsaid at least one extension sub-block can be transmitted; and a seconddata processor comprising: receiving means for receiving a basicsub-block and at least one extension sub-block transmitted from thefirst data processor; reproducing means for reproducing data from thesub-blocks received by the receiving means; and reproduction controllingmeans for reproducing data, if the sub-block received is determined tobe a basic sub-block, by supplying the basic sub-block and said at leastone extension sub-block received by the receiving means before asubsequent basic sub-block is received by the receiving means to thereproducing means.
 2. A data processing method comprising the steps of:generating a basic sub-block essential for reproducing original data andat least one extension sub-block essential for improving the quality ofthe reproduced data by dividing the original data into a plurality ofblocks, further dividing each of the plurality of blocks into aplurality of frequency bands, and then compressing the plurality offrequency bands; and transmitting a basic sub-block of a predeterminedtime unit, determining, after transmitting the basic sub-block, whethersaid at least one extension sub-block of the current block can betransmitted before a basic sub-block of a subsequent block istransmitted, and transmitting said at least one extension sub-block ifit has been determined that said at least one extension sub-block can betransmitted.
 3. The data processing method according to claim 2 whereinidentification information for identifying the sub-blocks is added toeach of the sub-blocks.
 4. The data processing method according to claim3 wherein the identification information includes information indicatingthe data size of the sub-blocks.
 5. The data processing method accordingto claim 3 wherein the identification information includes informationindicating whether the sub-block is a basic sub-block or an extensionsub-block.
 6. The data processing method according to claim 2 whereineach frequency band is compressed to generate a compression componentand a compression error component, the basic sub-block is thecompression component of the lowest frequency band, and said at leastone extension sub-block is the compression error components of thelowest frequency band and the compression components and the compressionerror components of the frequency bands other than the lowest frequencyband.
 7. The data processing method according to claim 6 wherein acompression error component of a frequency band is recompressed togenerate a recompression component and a recompression error component,and the recompression component and the recompression error componentare extension sub-blocks.
 8. The data processing method according toclaim 2 wherein at least one extension sub-block to be transmitted isselected from a plurality of the extension sub-block.
 9. The dataprocessing method according to claim 2 wherein, after a basic sub-blockis transmitted, at least one extension sub-block generated from the sameblock as the basic sub-block is transmitted if a transmission timing fortransmitting a sub-block is detected before a basic sub-block of asubsequent block is transmitted.
 10. A data processor comprising:transmission data generating means for preparing a basic sub-blockessential for reproducing original data and at least one extensionsub-block essential for improving the quality of the reproduced data bydividing the original data into a plurality of blocks, further dividingeach of the plurality of blocks into a plurality of frequency bands, andthen compressing the plurality of frequency bands; transmission meansfor transmitting the generated sub-blocks; and transmission controllingmeans for transmitting a basic sub-block of a predetermined time unit,determining, after transmitting the basic sub-block, whether said atleast one extension sub-block of the current block can be transmittedbefore a basic sub-block of a subsequent block is transmitted, andtransmitting said at least one extension sub-block if it has beendetermined that said at least one extension sub-block can betransmitted.
 11. The data processor according to claim 10 wherein thetransmission data generating means adds identification information foridentifying the sub-blocks to each of the sub-blocks.
 12. The dataprocessor according to claim 11 wherein the identification informationincludes information indicating the data size of the sub-blocks.
 13. Thedata processor according to claim 11 wherein the identificationinformation includes information indicating whether the sub-block is abasic sub-block or an extension sub-block.
 14. The data processoraccording to claim 10 wherein each frequency band is compressed togenerate a compression component and a compression error component, thebasic sub-block is the compression component of the lowest frequencyband, and said at least one extension sub-block is the compression errorcomponents of the lowest frequency band and the compression componentsand the compression error components of the frequency bands other thanthe lowest frequency band.
 15. The data processor according to claim 14wherein a compression error component of a frequency band isrecompressed to generate a recompression component and a recompressionerror component, and the recompression component and the recompressionerror component are extension sub-blocks.
 16. The data processoraccording to claim 10 wherein at least one extension sub-block to betransmitted is selected from a plurality of the extension sub-block. 17.The data processor according to claim 10 further comprising: a timinggeneration means for generating a timing for transmitting a basicsub-block or an extension sub-block, wherein the transmissioncontrolling means transmits a basic sub-block when the timing fortransmitting a basic sub-block was detected and an extension sub-blockgenerated from the same block as the basic sub-block is transmitted if atransmission timing for transmitting a sub-block is detected before abasic sub-block of a subsequent block is transmitted.
 18. A dataprocessing program written in a language readable by a computer so thatdata can be processed on a computer system, comprising the steps of:dividing the data into a plurality of blocks of a predetermined timeunit; generating a basic sub-block essential for reproducing originaldata and at least one extension sub-block essential for improving thequality of the reproduced data by dividing the original data into aplurality of blocks, further dividing each of the plurality of blocksinto a plurality of frequency bands, and then compressing one of theplurality of frequency bands; and transmitting a basic sub-block of apredetermined time unit, determining, after transmitting the basicsub-block, whether said at least one extension sub-block of the currentblock can be transmitted before a basic sub-block of a subsequent blockis transmitted, and transmitting said at least one extension sub-blockif it has been determined that said at least one extension sub-block canbe transmitted.
 19. A data processing method comprising the steps of:receiving a basic sub-block or at least one extension sub-block; andreproducing data from the basic sub-block and said at least oneextension sub-block received before a subsequent basic sub-block isreceived.
 20. The data processing method according claim 19 wherein atleast one extension sub-block is selected from a plurality of theextension sub-blocks received before a subsequent basic sub-block isreceived and data is reproduced from the received basic sub-block andselected said at least one extension sub-block.
 21. A data processorcomprising: receiving means for receiving a basic sub-block and at leastone extension sub-block; reproducing means for reproducing data from thesub-blocks received by the receiving means; and reproduction controllingmeans for reproducing data, if the sub-block received is determined tobe a basic sub-block, by supplying the basic sub-block and said at leastone extension sub-block received by the receiving means before asubsequent basic sub-block is received by the receiving means to thereproducing means.
 22. The data processor according to claim 21 whereinat least one extension sub-block is selected from a plurality of theextension sub-blocks received by the receiving means before a subsequentbasic sub-block is received and data is reproduced by supplying theselected said at least one extension sub-block and the basic sub-blockto the reproducing means.
 23. A data processing program written in alanguage readable by a computer so that data can be processed on acomputer system, comprising the steps of: determining whether thereceived sub-block is a basic sub-block; and reproducing data, if thereceived sub-block is a basic sub-block, from the basic sub-block and atleast one extension sub-block received before a subsequent basicsub-block is received.